CNN U-Net for classifiying tree 3 [Binary Classifier]¶

Repo: https://github.com/ayoubft/CNN-rgbUAV

We will be classifying a specific tree.

image.png

We will be using the whole image because we need the results to stay georeferenced so we can interpret the results visually.

image.png

Adapted U-net CNN-architecture for the tree species segmentation Ronneberger et al., 2015. This scheme illustrates how 128 × 128 pixel tiles were analyzed. Values on top of the boxes depict the number of calculated feature maps with the respective x-y-dimensions as vertically oriented labels.Ref

image.png

Importing the necessary libraries that will allow us to:

  • os: allow interactions with the os (creating and removing a directory (folder), fetching its contents, changing and identifying the current directory, etc... Link

  • PIL: Python Imaging Library is the de facto image processing package for Python language. It incorporates lightweight image processing tools that aids in editing, creating and saving images. Link

  • cv2: Pre-built CPU-only OpenCV(OpenCV (Open Source Computer Vision Library is an open source computer vision and machine learning software library.) packages for Python. Link

  • numpy: NumPy is very useful for performing mathematical and logical operations on arrays. It provides an abundance of useful features for operations on n-arrays and matrices in Python ... Link

  • matplotlib: Matplotlib is a complete library for creating static, animated and interactive visualizations in Python. Matplotlib makes the easy things easy and the hard things possible. Link

  • tensorflow: TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications. Link

In [1]:
import os
from PIL import Image
import cv2
import glob
import numpy as np
from matplotlib import pyplot as plt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, UpSampling2D

! Be careful to uncomment next line cropped images will be deleted

In [2]:
!sh clearCroppingDir.sh

Here we will crop the two pictures of training and testing, and their respective target images; into little tiles of size (256, 256).

In [3]:
def crop_images(savedir, filename, save_name):
    img = Image.open(filename)
    width, height = img.size
    w, h = (256, 256)

    frame_num = 1
    for col_i in range(0, width, w):
        for row_i in range(0, height, h):
            crop = img.crop((col_i, row_i, col_i + w, row_i + h))
            save_to = os.path.join(savedir, save_name + "_{:03}.jpg")
            crop.save(save_to.format(frame_num))
            frame_num += 1


crop_images("./Cropped/test", "./Data/test-Z2t3.tif", "test")
crop_images("./Cropped/train", "./Data/train-Z2t3.tif", "train")
crop_images("./Cropped/target-test", "./Data/target-test-Z2t3.tif", "target-test")
crop_images("./Cropped/target-train", "./Data/target-train-Z2t3.tif", "target-train")

We will be sorting the variables containing names of each tile.

In [4]:
files_test = glob.glob("./Cropped/test/*.jpg")
files_test.sort()
files_train = glob.glob("./Cropped/train/*.jpg")
files_train.sort()
files_tar_test = glob.glob("./Cropped/target-test/*.jpg")
files_tar_test.sort()
fileTest = glob.glob("./Data/test-Z2t3.tif")
fileTarTest = glob.glob("./Data/target-test-Z2t3.tif")
files_tar_train = glob.glob("./Cropped/target-train/*.jpg")
files_tar_train.sort()
In [5]:
files_test[:10]
Out[5]:
['./Cropped/test/test_001.jpg',
 './Cropped/test/test_002.jpg',
 './Cropped/test/test_003.jpg',
 './Cropped/test/test_004.jpg',
 './Cropped/test/test_005.jpg',
 './Cropped/test/test_006.jpg',
 './Cropped/test/test_007.jpg',
 './Cropped/test/test_008.jpg',
 './Cropped/test/test_009.jpg',
 './Cropped/test/test_010.jpg']
In [6]:
fileTarTest
Out[6]:
['./Data/target-test-Z2t3.tif']

Here are the tiles (cropped from original)

Voici image image.png

Create the X and y variables that will host respectively the tiles of training and testing.

In [7]:
X = []
for myFile in files_train:
    image = cv2.imread(myFile)
    X.append(image)

X = np.array(X)
print('X:', X.shape)

X: (357, 256, 256, 3)
In [8]:
y = []
for myFile in files_tar_train:
    image = cv2.imread(myFile)
    y.append(image)

y = np.array(y)
print('y:', y.shape)

y: (357, 256, 256, 3)

This a reminder of the architecture of the CNN U-Net that we will be using. Ref

image.png

In [9]:
# Construction d'un RNA convolutif
CNN = Sequential([Conv2D(64, (3, 3), padding='same', activation='relu', input_shape=(256, 256, 3)),
                  Conv2D(64, (3, 3), padding='same', activation='relu'),
                  MaxPooling2D((2, 2)),
                  Conv2D(128, (3, 3), padding='same', activation='relu'),
                  Conv2D(128, (3, 3), padding='same', activation='relu'),
                  MaxPooling2D((2, 2)),
                  Conv2D(256, (3, 3), padding='same', activation='relu'),
                  Conv2D(256, (3, 3), padding='same', activation='relu'),
                  MaxPooling2D((2, 2)),
                  Conv2D(512, (3, 3), padding='same', activation='relu'),
                  Conv2D(512, (3, 3), padding='same', activation='relu'),
                  MaxPooling2D((2, 2)),
                  Conv2D(1024, (3, 3), padding='same', activation='relu'),
                  UpSampling2D(size=(2, 2)),
                  Conv2D(512, (3, 3), padding='same', activation='relu'),
                  Conv2D(512, (3, 3), padding='same', activation='relu'),
                  UpSampling2D(size=(2, 2)),
                  Conv2D(256, (3, 3), padding='same', activation='relu'),
                  Conv2D(256, (3, 3), padding='same', activation='relu'),
                  UpSampling2D(size=(2, 2)),
                  Conv2D(128, (3, 3), padding='same', activation='relu'),
                  Conv2D(128, (3, 3), padding='same', activation='relu'),
                  UpSampling2D(size=(2, 2)),
                  Conv2D(64, (3, 3), padding='same', activation='relu'),
                  Conv2D(64, (3, 3), padding='same', activation='relu'),
                  Conv2D(3, (1, 1), activation='softmax')])
CNN.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 256, 256, 64)      1792      
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 256, 256, 64)      36928     
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 128, 128, 64)      0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 128, 128, 128)     73856     
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 128, 128, 128)     147584    
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 64, 64, 128)       0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 64, 64, 256)       295168    
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 64, 64, 256)       590080    
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 32, 32, 256)       0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 32, 32, 512)       1180160   
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 32, 32, 512)       2359808   
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 16, 16, 512)       0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 16, 16, 1024)      4719616   
_________________________________________________________________
up_sampling2d (UpSampling2D) (None, 32, 32, 1024)      0         
_________________________________________________________________
conv2d_9 (Conv2D)            (None, 32, 32, 512)       4719104   
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 32, 32, 512)       2359808   
_________________________________________________________________
up_sampling2d_1 (UpSampling2 (None, 64, 64, 512)       0         
_________________________________________________________________
conv2d_11 (Conv2D)           (None, 64, 64, 256)       1179904   
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 64, 64, 256)       590080    
_________________________________________________________________
up_sampling2d_2 (UpSampling2 (None, 128, 128, 256)     0         
_________________________________________________________________
conv2d_13 (Conv2D)           (None, 128, 128, 128)     295040    
_________________________________________________________________
conv2d_14 (Conv2D)           (None, 128, 128, 128)     147584    
_________________________________________________________________
up_sampling2d_3 (UpSampling2 (None, 256, 256, 128)     0         
_________________________________________________________________
conv2d_15 (Conv2D)           (None, 256, 256, 64)      73792     
_________________________________________________________________
conv2d_16 (Conv2D)           (None, 256, 256, 64)      36928     
_________________________________________________________________
conv2d_17 (Conv2D)           (None, 256, 256, 3)       195       
=================================================================
Total params: 18,807,427
Trainable params: 18,807,427
Non-trainable params: 0
_________________________________________________________________

Comiling the model using rmsprop optimizer, binary_crossentropy as loss function, and accuracy as metric.

In [10]:
CNN.compile(optimizer='rmsprop',
            loss='binary_crossentropy',
            metrics=['accuracy'])

Fitting the model using a 0.3 validation split, and 5 epochs.

In [11]:
history_CNN = CNN.fit(X,
                      y,
                      verbose=True,
                      epochs=5,
                      validation_split=0.3)

Epoch 1/5
8/8 [==============================] - 676s 85s/step - loss: 7476554.0000 - accuracy: 0.0119 - val_loss: 0.6815 - val_accuracy: 0.0510
Epoch 2/5
8/8 [==============================] - 659s 83s/step - loss: 1.2160 - accuracy: 0.1673 - val_loss: 2.0947 - val_accuracy: 0.0321
Epoch 3/5
8/8 [==============================] - 647s 81s/step - loss: 1.1045 - accuracy: 0.4807 - val_loss: 0.6416 - val_accuracy: 0.0792
Epoch 4/5
8/8 [==============================] - 675s 86s/step - loss: 0.6929 - accuracy: 0.2344 - val_loss: 0.6334 - val_accuracy: 0.0881
Epoch 5/5
8/8 [==============================] - 690s 87s/step - loss: 0.6472 - accuracy: 0.3937 - val_loss: 0.6512 - val_accuracy: 0.9813
In [12]:
history_CNN.history
Out[12]:
{'loss': [7476554.0,
  1.216025948524475,
  1.1045079231262207,
  0.6928532123565674,
  0.6472128629684448],
 'accuracy': [0.011913375928997993,
  0.16728056967258453,
  0.4807279706001282,
  0.2343672215938568,
  0.3937111496925354],
 'val_loss': [0.6814900040626526,
  2.0947165489196777,
  0.64158695936203,
  0.6333819031715393,
  0.6511528491973877],
 'val_accuracy': [0.05098795145750046,
  0.032127946615219116,
  0.07918830960988998,
  0.08806652575731277,
  0.9812894463539124]}

Testing:

In [13]:
Xtest = []
for myFile in fileTest:
    image = cv2.imread(myFile)
    Xtest.append(image)

Xtest = np.array(Xtest)

ytest = []
for myFile in fileTarTest:
    image = cv2.imread(myFile)
    ytest.append(image)

ytest = np.array(ytest)
In [14]:
y_new = CNN.predict(Xtest)

WARNING:tensorflow:Model was constructed with shape (None, 256, 256, 3) for input KerasTensor(type_spec=TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='conv2d_input'), name='conv2d_input', description="created by layer 'conv2d_input'"), but it was called on an input with incompatible shape (None, 1673, 4279, 3).
In [15]:
np.unique(y_new).max()
Out[15]:
0.34366682
In [20]:
y_new.shape
Out[20]:
(1664, 4272)

All nan ??

In [37]:
fig, ax = plt.subplots(figsize=(14.5, 8))
pos = ax.imshow(y_new, interpolation='nearest')

image.png

image.png

image.png

As we can see the results are far away from desired output, and this is because we have so little data. We will fit our model on a large dataset soon.